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INTRODUCTION 


Recent  developments  in  aircraft  technology  have  resulted  in  increased 
firepower  and  tactical  flexibility  during  air-to-ground  missions. 

However,  these  improvements  have  not  been  introduced  without  some  cost 
to  the  aircrew  by  increasing  their  workload  and  decreasing  the  time  in 
which  they  have  to  make  decisions.  This  report  is  part  of  a  Navy  pro¬ 
gram  whose  goal  is  to  look  at  the  feasibility  of  alleviating  aircrew 
overloading  by  using  computerized  decision  aids.  The  purpose  of  these 
aids  is  not  to  automate  decision-making,  but  rather  to  structure  the 
decision  task  in  such  a  way  as  to  reduce  the  operator's  processing  load. 

SCOPE 


This  report  is  a  review  of  state-of-the-art  decision  aids  being 
developed  for  various  military  applications.  These  aids  address  quite 
specific  problems,  some  of  which  are  unrelated  to  air-to-ground  missions. 
However,  the  mathematical  models  and  algorithms  used  to  generate  these 
aids  are  general  in  scope.  Therefore,  the  general  approaches  used  by 
the  developers  of  these  aids  may  prove  profitable  as  aiding  techniques 
in  the  fighter /attack  aircraft  environment. 

Currently,  a  great  number  of  decision  aids  are  being  developed  for 
military  use.  Because  of  this,  the  review  focuses  on  only  five  aids. 

This  allows  each  aid  to  be  described  in  detail  while,  at  the  same  time, 
the  diverse  procedural  approaches  used  by  the  various  aids  can  be  demon¬ 
strated.  The  first  section  of  the  report  will  describe  general  features 
of  decision-aiding.  This  will  form  a  basis  for  comparing  the  aids.  As 
part  of  this  effort,  a  taxonomy  of  decision-aiding  will  be  developed. 

The  purpose  of  the  taxonomy  will  be  to  accent  both  differences  and 
similarities  among  the  aids.  The  latter  sections  of  the  report  will  be 
devoted  to  describing  the  individual  aids. 

This  report  will  be  combined  with  information  from  two  other  reports: 

(1)  a  report  identifying  aircrew  decisions  during  attack  missions,1  and 

(2)  a  review  of  the  operator's  general  decision-making  characteristics. 2 
This  combination  will  lead  to  some  specific  recommendations  for  demon¬ 
strating  the  feasibility  of  using  a  computer-generated  decision  aid  in 
attack  aircraft. 


'Naval  Weapons  Center.  Identification  of  Significant  Aircrew 
Decisions  in  Navy  Aircraft ,  by  J.  Saleh  et  al.  Perceptronics,  Inc., 
China  Lake,  CA,  NWC,  June  1979.  (NWC  TP  6117,  publication  UNCLASSIFIED.) 

zNaval  Weapons  Center.  Operator  Decision-Making  Characteristics , 
by  Michael  J.  Barnes.  China  Lake,  CA,  NWC,  July  1979.  (NWC  TP  6124, 
publication  UNCLASSIFIED.) 
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PRINCIPLES  OF  DECISION-AIDING:  OVERVIEW 

The  aids  which  are  discussed  have  been  constructed  using  certain 
general  design  principles.  This  section  is  a  review  of  the  current 
principles  which  drive  this  design  process.  The  research  comes  from  two 
distinct  disciplines:  (1)  psychology,  whose  domain  concerns  the  pro¬ 
blems  associated  with  interfacing  the  aid  and  the  operator;  (2)  decision 
analysis,  which  focuses  on  the  mathematical  rigor  of  the  models  on  which 
the  aids  are  based. 

Research  dealing  with  the  interface  problem  has  resulted  in  re¬ 
defining  the  purpose  of  decision-aiding.  It  was  found  that  aids  which 
replaced  the  operator  during  important  decisions  resulted  in  system 
degradation.3’4'5  This  led  to  the  current  emphasis  on  constructing 
aids  whose  purpose  was  to  aid  rather  than  replace  the  operator.  Kelly6 
perhaps  best  characterized  the  current  thinking  on  the  purpose  of 
decision-aiding.  It  should  be  "a  mechanism  that  extends  the  intellect 
of  the  decision  maker  and  puts  structure  to  what  would  otherwise  be  a 
confusing  situation,  rather  than  a  device  which  makes  decisions  accord¬ 
ing  to  preset  rules  which  are  programmed  without  any  form  of  inter¬ 
vention  by  the  decision  maker."  This  characterization  implies  three 
criteria  for  a  good  aid: 

1.  It  brings  structure  to  the  decision  situation. 

2.  It  performs  functions  which  would  otherwise  overload  the 
operator's  processing  capacity. 

3.  It  leaves  the  final  decision  to  the  operator. 


3Halpin,  S.  M.  ,  E.  M.  Johnson,  and  J.  A.  Thornberry.  "Cognitive 
Reliability  in  Manned  Systems,"  IEEE  Trims.  Reliability  ,  Vol.  R-22, 
pp.  165-170  (1973). 

4U.  S.  Army  Research  Institute  for  the  Behavioral  and  Social 
Sciences.  Development  and  Application  of  a  Decision  Aid  for  Tactical 
Control  of  Battlefield  Operations :  A  Conceptual  Structure  for  Decision 
Support  in  Tactical  Operational  Systems ,  by  R.  Levitt  et  al. ,  Honeywell 
Inc.  Arlington,  VA,  1977.  (Technical  Report  TR-77-A2,  publication 
UNCLASSIFIED.) 

^Smithsonian  Institution.  Operational  Decision  Aids:  A  Program 
of  Applied  Research  for  Naval  Command  and  Control  Systems,  by  H.  Wallace 
Sinaiko.  Washington,  DC,  1977.  (Technical  Report  No.  5,  publication 
UNCLASSIFIED.) 

6Decisions  and  Designs,  Inc.  Decision  Analysis  and  Decision  Aids, 
by  C.  W.  Kelly.  McLean,  VA.  (In-house  paper,  publication  UNCLASSIFIED.) 


1 4 


NWC  TP  6171 


In  general,  the  reviewed  aids  meet  these  criteria.  More  detailed 
principles  of  aiding  have  been  developed.7 

1.  Systems  requirements  for  the  system  to  be  aided  must  be  well- 
defined  and  explained  in  detail. 

2.  The  output  of  the  aid  should  address  these  requirements. 

3.  The  displayed  output  should  be  human-factored  in  order  to 
present  information  in  an  optimal  manner. 

4.  The  displayed  information  should  be  in  sufficient  detail  to 
allow  the  operator  to  understand  the  state  of  the  system  being  aided. 

5.  The  aid  should  not  display  too  much  information  (which  could 
overload  the  operator) . 

6.  The  aid  should  be  designed  with  user  acceptance  in  mind. 

This  last  principle  is  the  crux  of  many  problems  that  system 
designers  have  had  with  introducing  new  aids.  No  matter  how  sophisti¬ 
cated  the  aid,  it  is  worthless  if  it  is  not  accepted  by  the  users. 

The  best  solution  to  this  problem3’7  seems  to  be  to  include  potential 
users  in  the  design  process,  during  both  development  and  testing  of  the 
aid.  As  a  corollary  to  this  principle,  the  training  level  necessary  to 
use  the  aid  should  be  commensurate  with  the  skill  level  of  the  intended 
user  population. 

It  would  be  an  advantage  if  any  of  these  criteria  could  be  used  to 
evaluate  aids.  Too  often,  however,  there  is  not  an  objective  measure  of 
system  performance  for  the  complex  systems  the  aids  were  designed  for 
(e.g.,  command  and  control  environment).  Many  of  the  reviewed  aids  are 
in  the  process  of  being  tested,  but  the  evaluation  procedure  remains 
somewhat  controversial.5’8’9  For  this  reason,  the  present  report  is  a 
description  of  what  the  aids  are  designed  to  do  and  the  mathematical 
models  the  aids  are  based  on.  Evaluation  of  the  aids,  except  in  the 
most  general  terms,  is  beyond  the  scope  of  this  report. 

7System  Planning  Corporation.  An  Investigation  of  Operational 
Decision  Aids ,  by  G.  L.  Lucas  and  J.  A.  Ruff.  Arlington,  VA,  1977. 

(Report  312,  publication  UNCLASSIFIED.) 

9 Analytics.  Measuring  the  Performance  of  Operational  Decision 
Aids.  Willow  Grove,  PA,  1976.  (Final  Report  1161-B. ) 

9Naval  Personnel  Research  and  Development  Center.  Significance 
of  Risk  in  Navy  Tactical  Decision  Making:  An  Empirical  Investigation , 
by  C.  Gettys  et  al.  San  Diego,  CA.  (NPRDC-TR-77-8,  publication 
UNCLASSIFIED.) 
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DECISION  TAXONOMY 


A  decision  taxonomy  was  developed  for  this  review  to  illustrate  the 
general  features  of  the  various  aids.  Other  taxonomic  breakdowns  for 
decisions  were  found  to  be  either  too  specific  or  too  detailed  for  the 
purposes  of  this  report.10*11’12  The  scope  of  the  present  taxonomy  is 
limited  to  tasks,  approaches,  and  conditioning  elements  which  would 
generalize  to  any  decision  category.  The  first  part  of  the  taxonomy  is 
a  breakdown  of  generic  decision  tasks.  The  aids  themselves  are  defined 
in  terms  of  the  decision  tasks  they  were  designed  to  aid. 


DEFINITION  OF  AN  AID 

Unless  the  definition  of  a  decision  aid  is  limited  to  specific 
tasks,  the  term  itself  becomes  all-embracing.  This  is  because  all 
displays  are  designed  to  improve  some  aspect  of  the  operator's  decision¬ 
making  performance.  What  separates  decision  aids  from  other  displays  is 
the  specialized  intent  of  the  decision  aid's  output.  Historically,  aids 
designated  as  decision  aids  displayed  information  aiding  the  operator  in 
such  specialized  decision  tasks  as  probability  estimation  or  utility 
assessment.13’14  With  this  in  mind,  a  decision  aid  is  an  algorithm  or 
mathematical  procedure  whose  output  structures  or  generates  information 
relating  to  some  decision  task.  The  particular  tasks  defined  as 
decision  tasks  are  discussed  below. 

DECISION  TASK  TAXONOMY 

The  taxonomic  breakdown  presented  here  is  not  particularly  original. 
It  is  a  synthesis  of  other  taxonomic  breakdowns  that  attempt  to  be 
complete  without  being  overwhelming  in  detail.  For  more  detail,  the 
original  papers  contain  in-depth  discussions  of  the  decision  tasks 
reviewed.2’*2’15’16 


10Decision  and  Designs  Inc.  Selecting  Analytic  Approaches  for 
Decision  Situations:  A  Matching  of  Taxonomies ,  by  R.  Brown  and 
J.  Ulvila.  McLean,  VA,  October  1976.  (TR-76-10,  publication  UNCLAS¬ 

SIFIED.) 

^Applied  Decision  Analysis,  Inc.  An  Analytic  Characterization  of 
Navy  Command  and  Control  Decisions ,  by  A.  Miller  et  al.  Menlo  Park,  CA, 
March  1979.  (Final  Report  3-31-79,  publication  UNCLASSIFIED.) 

12Naval  Training  and  Equipment  Center.  Decision  Making  and  Train¬ 
ing ,  by  R.  S.  Nickerson  and  G.  Feehrer.  Orlando,  FL,  NTEC,  1975. 
(Technical  Report  73-C-0128-1,  publication  UNCLASSIFIED.) 

13Edwards,  W. ,  "Dynamic  Decision  Theory  and  Probabilistic  Informa¬ 
tion  Processing,"  Human  Factors ,  Vol.  4,  pp.  59-73,  1962. 

1 4The  Rand  Corporation.  Judge:  A  Value-Judgment-Based  Tactical 
Command  System ,  by  L.  Miller  et  al.  Santa  Monica,  CA,  March  1967. 
(RM-5147-PR,  publication  UNCLASSIFIED.) 
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Two  distinct  processes  underlie  most  decision  analysis:  the 
decision  situation  is  structured,  and  mathematical  rules  are  computed  to 
select  among  decision  alternatives.17  These  two  processes,  structuring 
and  selection,  form  the  basis  of  at  least  one  other  decision  taxonomy.1* 
For  the  present  taxonomy,  this  division  into  problem  structuring  and 
alternative  selection  dichotomizes  decision-making  into  two  classes 
of  decision  tasks.  The  taxonomy  represents  ideal  decision-making, 
reflecting  how  the  operator  should  make  decisions,  not  necessarily  how 
he  does.  These  decision  tasks  are  listed  in  Table  1. 

Problem  structuring  is  probably  the  most  difficult  part  of  decision 
making.  It  requires  a  detailed  expertise  in  the  parameters  of  the  real 
world  environment  as  well  as  expertise  in  decision  structuring.  The 
pitfalls  in  such  an  analysis  are  many,  and  the  reader  should  not  be 
mislead  into  thinking  that  the  problem  is  trivial  by  the  simplified 
example  given  below  (for  insight  into  the  complexities  involved  see 
Fischhoff,  1980;  Howard,  Matheson,  and  North,  1977). 18,19  Figure  1 
illustrates  a  decision  tree  approach  to  structuring  a  problem  wherein 
two  actions  and  three  states-of-the-world  are  possible.  The  operator 
first  decides  what  action  options  he  has  (A  or  B) .  Next,  he  must  create 
an  hypothesis  set  representing  distinct  states-of-the-world  he  feels  are 
possible  at  the  conclusion  of  each  of  the  actions  (H^,  H2,  or  H^) .  His 

last  task,  in  our  simplified  example,  is  to  generate  a  list  of  outcomes. 
Outcomes  are  the  consequences  of  each  action  given  a  particular  state- 
of-the-world  (i.e.  ,  Aft  ^  *  U,  » ,  BflH^  *  Z) . 

Figure  2  depicts  four  steps  in  the  alternative  selection  process. 
Basically,  these  tasks  consist  of  computing  certain  values  in  order  to 
generate  a  decision  rule  with  which  to  choose  an  optimal  course  of 
action  after  the  decision  problem  is  structured. 


15Keeney,  R.  L. ,  and  H.  Raiffa.  Decision  with  Multiple  Objectives: 
Preferences  and  Value  Trade-offs .  New  York,  John  Wiley  &  Sons,  1976. 

549  pp. 

lsNaval  Training  Equipment  Center.  Analysis  of  Requirements  and 
Methodology  for  Decision  Training  in  Operational  Systems ,  by  J.  Saleh, 
et  al.  Orlando,  FL,  NT EC,  1978.  (TR-77-0055-1,  publication  UNCLASSIFIED.) 

17Decision  and  Designs,  Inc.  Handbook  for  Decision  Analysis: 
Structure  of  a  Decision  Tree ,  by  C.  R.  Peterson,  et  al.  McLean,  VA, 

October  1973. 

18Fischhoff,  B.  For  Those  Condemned  to  Study  the  Past:  Reflections 
on  Historial  Judgment  in  Hew  Directions  for  Methodology  of  Behavioral 
Sciences  Fallible  Judgment  in  Behavioral  Research ,  eds.  R.  Shweder  and 
D.  Fiske.  San  Francisco,  Jossey-Bass  Co.,  1980. 

19Howard,  R.  A.,  J.  Matheson,  and  J.  E.  North.  "The  Decision  to 
Seed  Hurricanes,"  Science,  Vol.  176,  1972,  pp.  1191-1202. 
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TABLE  1.  Listing  of  Decision  Tasks. 


I.  PROBLEM  STRUCTURING 

a.  List  possible  actions 

b.  Generate  hypothesis 

c.  Determine  outcomes 


II.  ALTERNATIVE  SELECTION 

a.  Information  selection 

1.  Information  gathering 

2.  Optional  stopping  of  information  gathering 

b.  Probability  estimation 

1.  Event  estimation  (known  distribution) 

2.  Event  estimation  (unknown  distribution) 

3.  Update  probability  (conditional  independence) 

U.  Update  probability  (conditional  dependence) 

5.  Multistage  inference 

6.  Probabilistic  model  generation 

c.  Worth  assessment 

1.  Outcome  ranking 

2.  Outcome  interval  scaling 

3.  Multi-attribute  interval  scaling 

d.  Action  selection 

1.  Worth  selection 

2.  Expected  utility  selection 

3.  Other  rules 
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(1)  ACTIONS  <2>  STATES  OF  THE  WORLD 

H, 


(31  CONSEQUENCES 
OR  OUTCOMES 
IF  A  AND  H, 
THEN  U  OCCURS 


\H3 


IF  A  AND  H2 
THEN  V  OCCURS 


IF  A  AND  H3 
THEN  W  OCCURS 


IF  B  AND  H1 
THEN  X  OCCURS 


IF  B  AND  Hj 
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A.  B  -  SET  OF  POSSIBLE  ACTIONS  THI 

H..  H_,  H,  »  SET  OF  POSSIBLE  STATES- 
OF-THE-WORLD 

U,  V,  W,  X,  Y,  Z  -  SET  OF  OUTCOMES 

FIGURE  1.  Problem  Structuring  Using  a  Decision  Tree. 
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First,  information  must  be  sampled  to  compute  probabilities  for 
each  of  the  hypotheses.  The  second  step  is  the  actual  computation  of 
probabilities  based  on  sampled  data  and  prior  probability  information. 
The  third  step  is  an  evaluation  of  the  outcome  set.  This  consists  of 
numerically  assessing  the  worth  of  each  outcome.  The  last  step,  action 
selection,  involves  generating  the  actual  decision  rule  (usually  an 
equation)  used  to  choose  the  best  action.  The  parameters  of  the  rule 
are  based  ou  the  computations  from  both  probability  estimation  and  worth 
assessment.  However,  as  the  feedback  loop  in  Figure  2  indicates,  this 
process  is  iterative  in  nature.  Before  a  decision  is  made  to  choose  an 
action,  the  decision-maker  must  decide  whether  it  is  more  cost-effective 
to  act  or  to  sample  additional  information. 2 

The  sub^asks  for  each  of  these  components  of  alternative  selection 
are  discussed  next.  These  tasks  have  fairly  technical  definitions  and 
as  mentioned  before,  a  complete  discussion  of  these  tasks  is  contained 
elsewhere.2’12’15’16  Information  selection  tasks  are  either  the  first 
(subtask  1)  or  last  task  (subtask  2)  done  before  selecting  the  final 
decision.  The  probability  estimation  tasks  are  listed  in  order  of 
increasing  complexity  and  the  tasks  themselves  are  not  mutually  exclu¬ 
sive.  The  worth  assessment  tasks  are  defined  in  terms  of  the  scaling 
technique  used  to  generate  the  worth  functions.  Here  again  the  order  is 
from  least  to  most  sophisticated  technique.  Action  selection  tasks  are 
defined  in  terms  of  the  types  of  decision  rule  used. 


Information  Selection 


1.  Information  gathering  consists  of  deciding  which  data  sources 

to  sample,  depending  on  their  information  values  and  the  cost  of  sampling. 

2.  Optional  stopping  is  the  decision  as  to  whether  to  continue 
sampling  or  to  stop  sampling  and  choose  a  course  of  action. 


Probability  Estimation 

1.  Event  estimation  (known  distribution)  is  an  estimate  of  an 
event's  probability  given  only  current  information.  A  frequency  distri¬ 
bution  concerning  the  event's  occurrence  is  specified. 

2.  Event  estimation  (unknown  distribution)  is  the  same  task  except 
no  objective  frequency  distribution  concerning  the  event's  occurrence  is 
specified.  A  subjective  probability  distribution  must  be  generated. 

3.  Probability  updating  (conditional  independence)  is  an  event 
estimation  which  is  continually  updated  as  more  information  is  pro¬ 
cessed.  The  relationships  among  the  processed  data  are  mutually  con¬ 
ditionally  independent. 
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4.  Probability  updating  (conditional  dependence)  is  an  updating 
task  wherein  statistical  dependencies  exist  among  the  data  sources  used 
to  update  the  probability  estimate. 

5.  Multistage  inference  is  a  more  complex  event  estimation  task 
wherein  the  data  are  removed  from  the  hypothesis  set  by  two  or  more 
stages  of  intervening  probability  estimates  (also  known  as  hierarchical 
inferences) . 

6.  Probabilistic  model  generation  is  an  attempt  to  generate  a 
model  to  predict  the  trend  of  the  occurrence  of  future  events  (e.g., 
multiple  regression  model) . 

Worth  Assessment 

1.  The  outcomes  are  ranked  in  terms  of  their  relative  worth  or 
value  using  an  ordinal  scale. 

2.  A  worth  scale  with  interval  properties  (usually  a  utility 
scale)  is  defined  for  the  outcome  set. 

3.  The  outcomes  are  broken  down  into  independent  attributes 
(i.e.,  dimensions  of  worth)  and  modeled  so  that  the  resultant  functions 
constitute  an  interval  worth  scale. 


Action  Selection 

1.  The  decision  rule  consists  of  choosing  the  action  associated 
with  the  highest  worth  (i.e.,  no  probability  estimate  is  used  to  generate 
the  rule) . 

2.  The  decision  rule  is  based  on  choosing  the  action  with  the 
highest  expected  utility.20 

3.  All  other  decision  rules  such  as  minimax,  maximin,  lexico¬ 
graphic  ordering,  or  any  rule  not  mentioned  above. 

Z0Utility  is  a  measure  of  the  worth  of  an  outcome  in  an  uncertain 
environment  (see  Coombs,  Dawes,  and  Tversky,21  1970  for  a  precise 
mathematical  definition).  Expected  utility  is  the  expected  utility 
value  of  an  action  with  more  than  one  possible  outcome. 

21Coombs,  C.  H. ,  R.  Dawes,  and  A.  Tversky.  Mathematical  Psychology. 
Englewood  Cliffs,  Nd ,  Prentice  Hal],  Inc.,  1970.  419  pp. 
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Other  Characteristics 

Two  additional  features  of  the  aids  pertinent  to  this  classifica¬ 
tion  are:  the  conditioning  elements  implicitly  or  explicitly  assumed, 
and  the  mathematical  models  the  aids  are  based  on.  Conditioning  elements 
are  features  of  the  environment  such  as  weather  conditions  and  enemy 
strength  which  influence  both  probability  and  worth  values.  Condition¬ 
ing  elements  are  either  implicitly  assumed  in  computing  these  values  or 
explicitly  accounted  for  in  the  design  of  the  aid.  This  report  will 
focus  on  the  various  procedures  used  in  order  to  incorporate  condition¬ 
ing  elements  into  the  aiding  algorithms. 

In  comparing  these  aids,  it  will  be  evident  that  a  wide  variety  of 
mathematical  approaches  have  been  used.  Not  only  decision  theoretic 
models  such  as  Bayes  theorem  and  utility  theory,  but  other  models  based 
on  dynamic  and  non-linear  programming  and  discriminant  analysis  have 
been  used  to  aid  decision  situations.  Perhaps  even  more  interesting 
from  this  project's  point  of  view  are  some  of  the  diverse  uses  made  of 
the  same  general  approach.  The  next  section  is  a  review  of  various 
aiding  approaches.  The  last  section  summarizes  these  approaches  and 
compares  them. 


REVIEW  OF  INDIVIDUAL  AIDS 


Most  of  the  aids  discussed  were  developed  as  part  of  an  operational 
decision  aid  (ODA)  project  funded  by  the  Office  of  Naval  Research. 

These  aids  were  designed  mainly  as  research  tools  for  studying  decision¬ 
making.  However,  they  were  designed  to  be  used  in  realistic  command  and 
control  decision  environments.  A  number  of  military  scenarios  developed 
by  the  Stanford  Research  Institute  (SRI)  were  adapted  by  the  developers 
of  the  aids  to  describe  the  parameters  of  their  particular  environments. 

TACAID:  DECISIONS  AND  DESIGNS,  INC. 

TACAID  is  a  collection  of  aid  modules.  The  aid  is  flexible  and  has 
been  exercised  using  a  variety  of  combat  scenarios.  The  basic  display 
is  a  triangle  with  its  sides  measuring  three  probability  states  and  its 
interior  divided  into  three  or  more  action  regions.  The  boundaries  of 
the  action  regions  are  based  on  thresholds  derived  from  multi-attribute 
utility  models.  Other  factors  which  have  been  investigated  using  TACAID 
include  multiple  conditioning  elements,  risk  aversion  functions,  pre¬ 
diction  displays,  and  a  four-state  probability  model. 


if 
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Triangular  Display 

The  triangular  display's  purpose  is  to  show  the  relative  uncertainty  > 

of  three  possible  states  of  the  world  and  at  the  same  time  to  suggest  a 
course  of  action  to  the  commander.  Figure  3A  shows  the  skeleton  of  the 
triangle  to  illustrate  how  the  probability  information  is  displayed. 

Figure  3B  shows  a  display  as  the  operator  would  see  it  with  the  surface 
of  the  triangle  divided  into  three  regions  representing  the  military 
actions  the  commander  may  wish  to  consider.  The  sides  of  the  display 
indicate  possible  states  of  the  world  (S^,  S2,  or  S^)  which  influence 
the  commander's  choice  of  action. 

Figure  3A  shows  three  states  of  possible  enemy  intent:  routine 
surveillance  (SI),  probing  action  (S2) ,  or  full  scale  attack  (S3).  The 
relative  uncertainty  concerning  these  three  states  is  shown  on  the 
triangle's  surface  as  a  dot  with  a  circle  around  it.  This  encircled 
dot,  referred  to  as  a  probability  bug,  is  placed  at  the  intersection  of 
the  lines  projecting  from  the  sides  representing  each  of  the  probability 
states  (dark  lines  on  Figure  3B).  Thus  the  probability  bug  is  situated 
on  the  triangle  to  give  the  operator  a  pictorial  representation  of  the 
relative  uncertainty  for  each  of  the  three  possible  enemy  intents 
[P(S1)  =  0.2,  P(S2)  =  0.2,  and  P(S3)  =  0.6,  in  our  example]. 

The  actual  probability  estimates  are  input  parameters  to  the  display.  * 

An  intelligence  analyst  reports  specific  enemy  movements  which  influence 
these  estimates  (e.g.  ,  enemy  target  platform  deployed,  missile  signal 
report  ...).  Next,  an  algorithm  combines  the  new  reports  with  the  * 

present  estimate  of  these  events  [(P(S^),  P(S2) ,  or  P(S^))  using  Bayes 

theorem  to  update  the  three  probabilities.  Since  these  probabilities 
change  as  more  information  is  received,  the  probability  bug  moves  on 
the  surface  of  the  triangle  whenever  new  data  is  received. 

,  1 

Figure  3B  aids  the  commander  by  telling  him  what  the  suggested  j 

course  of  action  is.  Regions  of  the  display  are  separated  by  boundaries 
which  represent  probability  thresholds.  The  derivation  of  the  equation 
representing  the  boundary  is  based  on  a  multi-attribute  utility  model 
which  will  be  discussed  in  the  next  section.  Inituitively ,  the  regions 
represent  states  of  uncertainty  wherein  the  best  course  of  action  is 
explicated.  In  our  example,  the  probability  bug  indicates  there  is  a 
moderately  high  probability  that  the  enemy  is  engaging  in  a  full-scale 
attack  (P (S ^)  =  0.6)  and  smaller  probabilities  that  they  are  involved  in 

either  probing  actions  or  routine  surveillance  (P(S^)  =  P(S2)  =  0.2). 

For  this  situation,  the  TACAID  display  suggests  that  the  commander 
commit  a  large  force  for  counterattack.  The  boundaries  are  based  on 
values  elicited  from  the  commander  and  his  staff  before  the  start  of  the  • 

current  exercise.  Such  a  display  is  not  meant  to  automatically  commit  ; 

the  commander  to  a  course  of  action,  but  rather  to  suggest  a  normative 


14 


J 


NWC  TP  6171 


PRESENT  PROBABILITIES  FOR  EACH  0 


PROBABILITY  OF  ROUTINE  SURVEILLANCE 
STATE  1 


FIGURE  3A.  Probability  Plane  for  Three  Enemy  Intents. 
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solution  given  a  particular  sequence  of  indicators  from  the  intelligence 
analyst.  Since  the  bug  changes  position  as  more  data  is  received,  the 
display  also  allows  the  commander  to  visually  assess  the  changing  military 
environment.  Still,  this  general  approach  has  been  criticized,  since 
the  reasoning  behind  the  suggested  action  is  not  necessarily  transparent 
to  the  commander  and  might  tend  to  be  disregarded  when  the  consequences 
of  being  wrong  are  serious.  A  more  recent  TACAID  display  configuration 
(Figure  4)  has  three  triangles  each  suggesting  actions  depending  on 
three  separate  conditioning  elements:  strength  of  own  forces,  enemy 
strength,  and  weather  conditions.  The  rationale  for  the  display  is  that 
the  three  triangles  give  the  commander  a  better  feel  for  the  complexity 
that  underlies  a  military  environment.  It  also  gives  him  more  respon¬ 
sibility,  since  he  must  integrate  this  information  before  making  a 
decision.  As  mentioned  previously,  the  general  trend  in  aiding  is  to 
inform  the  operator  rather  than  select  actions  for  him. 

Mathematical  Model 

The  models  used  for  TACAID  were  developed  using  concepts  from 
Bayesian  decision  theory  and  utility  theory  (see  Savage  (1954)  for  a 
discussion  of  their  theoretical  basis).22  The  TACAID  is  divided  into 
action  regions  bounded  by  indifference  planes  called  probability  thres¬ 
holds.  Basically,  these  planes  are  equations  which  define  a  set  of 
probability  values  for  which  the  operator  is  theoretically  indifferent 
between  two  actions.  Table  2  illustrates  how  utility  values  are  used  to 
derive  decision  rules  which  define  action  regions.  These  tabulated 
values  are  examples  of  values  elicited  from  the  commander  to  represent 
the  relative  worth  (e.g.,  utility)  of  each  outcome.  In  our  example  the 
most  desirable  outcomes  for  each  action  are  assigned  a  zero  (on  the  top- 
bottom  diagonal  in  Table  2),  the  values  of  the  other  cells  are  negative 
showing  the  commander's  negative  assessments  using  zero  as  an  end 
point.  Using  these  values,  the  expected  utility  value  for  each  action 
is  computed  in  Step  1  (i.e.,  E(A^),  E(A2),  E(A^)).  Step  2  is  a  setting 

of  inequalities,  based  on  the  principle  that  Action  1  is  preferred  to 
Action  2  whenever  E(A^)  >  E(A2).  The  derivation  of  the  decision  rule 

involves  the  simplification  of  this  inequality  to  a  form  which  indicates 
that  one  of  the  probabilities  is  larger  than  the  weighted  combination  of 
the  other  two  probabilities.  Since  the  surface  of  the  TACAID  display  is 
defined  in  terms  of  the  relative  probabilities  of  three  states,  these 
inequalities  are  used  to  represent  non-overlapping  regions  on  the 
surface  of  the  display.  The  boundaries  of  these  regions  are  where  these 
inequalities  become  equations  or  more  formally  they  constitute  the 
planes  defined  by  those  probability  values  resulting  in  equal  expected 
value  for  at  least  two  of  the  actions. 


22Savage,  L.  J.  The  Foundation  of  Statist! < 

Wiley  &  Sons,  Inc.,  1954.  282  pp. 
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TABLE  2.  Derivation  of  Decision  Rules 
Using  Utility  Values. 


STATE S-OF-THE-WORLD 


S1 

S2 

S3 

A1 

0 

A 

B 

A2 

C 

0 

D 

A3 

E 

F 

0 

UTILITY 

VALUES 


where  A,  B,  C,  D,  E,  F<0 
and  B<D,  A<F,  C<E 

(1)  Expected  Utility 

E(A1)  =  P  (S2)  A  +  P(S3)B 
E (A2)  =  P(S1)C  +  P(S3)D 
E(A3)  =  P  (SI)  E  +  P(S2)F 

where  P(S1),  P(S2)  and  P(S3)  are 
probabilities  of  states  1,  2,  and 
3  occurring. 

(2)  Decision  Rules 

A1  >  A2  when  E(A1)  >  E(A2) 

P(S2)A  +  P(S3)B  >  P(S1)C  +  P(S3)D 

P(S1)  >  P(S2)|  +  P(S3)^ 

Ax  >  A3  when  P(S1)  >  P(S2)“  +  P(S3)| 

A2  >  A3  when  P(S2)  >  P(S1)~  +  P(S3)| 
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Decision  Aid  Modules 

The  triangular  display  is  only  part  of  the  decision  aiding  research 
done  for  the  TACAID  project.  However,  it  is  the  best  defined  aid  developed 
for  the  project  and  many  of  the  other  aids  listed  in  the  decision 
module  package  (Table  3)  are  actually  exploratory  concepts  rather  then 
developed  aids.  The  decision  modules  are  independent  aiding  procedures 
designed  to  aid  specific  generic  decision  tasks.  The  scope  of  decision 
tasks  addressed  is  wide  and  most  of  the  tasks  discussed  as  part  of  the 
taxonomy  (Table  1)  are  addressed  by  specific  modules.  The  rationale 
behind  the  modules  is  that  any  decision  situation  can  be  defined  in 
terms  of  the  specific  decision  subtasks  which  these  modules  are  designed 
to  aid.  Thus,  by  combining  various  modules  and  redefining  parameters, 
these  aids  theoretically  can  be  redesigned  to  fit  any  number  of  decision 


environments.  Of  course,  defining  the  system  requirements  for  specific 
decision  situations  and  developing  an  overall  algorithm  to  coordinate 
the  modules  may  prove  to  be  more  difficult  for  some  decision  environments 
than  starting  from  scratch  and  developing  new  aiding  approaches.  However 
approaches  using  algorithms  similar  to  some  of  the  modules  have  been 
used  recently  for  aiding  in  a  variety  of  decision  situations  by  the 
TACAID  developers  (cf.,  Kelly,  1978;  Amey,  Feuerwerger,  and  Gullck, 

1979).  Table  3  lists  the  modules,  decision  task  addressed,  decision 
type  (structuring  or  selection)  and  particular  decision  subtask  where 
appropriate.  The  modules  are  described  elsewhere  (Peterson,  Phillips, 
Randall,  and  Shaw  Cross,  1977).  The  following  descriptions  are  for 
modules  whose  functions  are  not  obvious  from  Table  3  and  may  be  important 
in  this  program. 

Invent  Course  of  Action.  This  module  is  an  attempt  to  use  a  reper¬ 
tory  grid  technique  to  generate  courses  of  action  not  considered  by  the 
operator.  Apparently  this  approach  was  still  in  the  early  exploratory 
stages  when  the  report  was  published. 

Process  Model.  This  module  is  an  information  display  giving  the 
operator  information  concerning  basic  processes  relating  to  the  system 
being  aided  (e.g.,  show  fuel  consumption  as  a  function  of  flight  time 
and  airspeed). 

Time-Discounting  Module.  This  module  shows  the  value  of  an  outcome 
if  it  takes  place  at  a  future  time  as  opposed  to  being  done  now. 

Influence  Model.  A  module  where  the  operator  estimates  probability 
values  for  parameters  which  influence  the  variable  of  interest.  An 
algorithm  estimates  the  probabilities  for  the  variables  of  interest 
based  on  these  inputs. 
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Sensitivity  Analysis.  A  range  of  values  is  given  for  either  utilities 
or  probabilities.  This  allows  the  operator  to  see  how  the  suggested 
course  of  action  changes  as  the  values  of  important  parameters  change. 

In  summary,  the  modular  approach  is  an  attempt  to  describe  decision 
environments  as  consisting  of  various  molecular  combinations  of  decision 
tasks.  Modules  are  designed  to  be  mapped  to  particular  combinations. 

Such  an  approach  is  still  in  the  developmental  stage. 


T AC AID  TAXONOMIC  BREAKDOWN 

Decision  Tasks.  The  decision  tasks  addressed  by  the  various  aids 
are  listed  in  Table  3.  Perhaps  the  most  impressive  feature  of  the 
TACAID  project  is  the  wide  variety  of  decision  analytic  techniques  used. 
It  remains  to  be  seen  whether  operational  aids  based  on  these  techniques 
prove  useful  in  a  realistic  environment. 

Conditioning  Elements.  TACAID  considers  approaches  to  using  three 
conditioning  elements  to  describe  the  state-of-the-world  either  as  a  com¬ 
posite  probability  or  as  separate  sources  of  information  (see  Figure  3). 

Mathematical  Models.  The  aids  are  based  on  Bayes  theorem  and 
utility  theory,  as  well  as  other  standard  statistical  models  used  in 
decision  analysis. 
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ADDAM:  PERCEPTRONICS,  INC. 

The  ADDAM  algorithm  is  a  versatile  aid  which  combines  a  decision 
theoretic  approach  with  techniques  used  in  pattern-recognition  research. 
The  operator  sees  a  list  of  suggested  actions  ranked  according  to  a 
multi-attribute  utility  (MAUT)  model.  Furthermore,  the  utility  values 
of  the  various  attributes  are  updated  as  a  function  of  the  operator's 
responses.  This  is  done  using  a  pattern-recognition  training  algorithm 
which  changes  values  of  outcomes  favored  by  the  chosen  response  by 
raising  their  utility  weights,  and  downgrades  those  with  low  levels. 

This  adjustment  of  weights  stabilizes  after  a  relatively  few  trials  if 
the  decision  environment  remains  stationary.  The  aid  has  been  adapted 
for  a  number  of  different  military  environments  including  a  command  and 
control  information  system,  remotely  piloted  vehicle  (RPV)  control  and 
as  an  aid  for  the  Tactical  Coordination  Officer  (TACCO)  during  anti¬ 
submarine  warfare  (ASW)  maneuvers. 


The  MAUT  Model  in  ADDAM 


For  the  MAUT  model,  consequences  of  an  action  are  broken  down  into 
its  constituent  parts  reflecting  independent  attributes  or  dimensions  of 
the  resulting  outcome.  The  value  of  an  outcome  (referred  to  as  its 
utility)  is  a  linear  function  of  these  attributes. 


n 

=  I 
i  =  1 


U .  a .  . 


where 

Vj  =  utility  value  of  the  Jth  outcome 

U^  =  weighting  constant  for  attribute  i 

a.,  =  value  of  attribute  i  for  the  Jth  outcome 
i 


(1) 
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The  value  of  a  response  Is  the  expected  value  of  the  possible  outcomes 
for  that  response. 


be  ■  A  PJ  VJ 


where 

R  =  value  of  Kth  response 
K 

Pj  =  probability  associated  with  the  Jth  outcome 
Vj  =  utility  value  of  Jth  outcome 

The  ADDAM  aid  displays  the  expected  value  of  various  responses 

(Rr . .R  . . .Rn)  to  the  operator.  Also,  probabilities  associated  with 

each  outcome  for  these  actions  can  be  displayed  to  the  operator.  The 
ADDAM  aid  seems  to  be  particularly  effective  for  repetitive  situations 
like  the  ASW  environment.  In  such  cases,  since  the  operator  must  make 
the  same  type  of  decision  a  number  of  times,  a  model  based  on  his  past 
performance  allows  him  to  be  consistent  and  reliable.  For  many  such 
tasks,  the  operator's  lack  of  consistency  degrades  performance.  Linear 
models  (such  as  Equations  1  and  2)  have  been  shown  to  outperform  the 
operator  they  model  in  a  wide  variety  of  decision  environments. 2 3  The 
next  section  indicates  how  attribute  weights  are  modified  by  the  operator's 
previous  responses. 


Pattern  Recognition  Model  in  ADDAM 


As  mentioned  before,  ADDAM  has  been  adapted  to  a  number  of  different 
environments.  The  following  discussion  involves  one  of  the  simpler 
models  used  to  update  utility  values  (V ^ . . . V ^ . . . V  ) .  However,  the 

general  principles  applicable  to  other  uses  of  the  ADDAM  model  are 
illustrated  by  this  example. 

Figure  5  outlines  the  general  pattern-recognition  approach.  Start¬ 
ing  on  the  left-hand  side  of  Figure  5,  one  can  see  that  each  possible 
response  (i.e.,  choice)  has  a  pattern  of  attributes  associated  with  it. 

On  the  right-hand  side  of  the  figure,  each  response  pattern  is  weighted 
and  summed  in  order  to  derive  a  utility  value  according  to  Equation  1. 

The  decision  selection  box  on  the  far  right  represents  the  selection  by 
the  ADDAM  algorithm  of  the  response  pattern  with  the  highest  utility 
value.  The  selected  response  is  referred  to  as  the  predicted  response. 
This  predicted  response  is  then  compared  to  the  operator's  actual 

'Dawes,  R.  M.  and  15.  Corrigan.  "Linear  Models  in  Decision-Making," 
Psychol.  Pull.,  Vol.  81  (1974),  pp.  95-106. 
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MULTIPLICATION  CELL 

FIGURE  5.  Outline  of  the  ADDAM  Algorithm  Including  Both  Multiattribute 
Utility  Selection  and  Pattern  Recognition  Error  Correction  Procedures. 
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derision  (lower  right  hand  corner) .  Next,  based  on  this  comparison, 
the  error-correction  training  procedure  readjusts  the  utility  weights 
for  Equation  1  (see  Figures  6  and  7).  The  attribute  levels  are  normal¬ 
ized  and  attributes  with  high  levels  on  the  chosen  response  compared  to 
the  predicted  response  result  in  increasing  the  weight  for  that  attri¬ 
bute.  Similarly,  relatively  low  attribute  levels  for  the  chosen  response 
result  in  lowering  the  weight  for  that  attribute.  Thus,  the  operator’s 
actual  response  pattern  determines  the  weighting  for  each  attribute  in 
Equation  1.  These  weighting  values  stabilize  in  a  relatively  short  time 
if  the  decision  environment  remains  constant.  Another  factor  which 
determines  the  time  it  takes  to  stabilize  is  the  value  of  X  shown  in 
Figure  7  which  is  called  the  adjustment  factor. 

Stabilization  is  said  to  occur  whenever  the  predicted  response  agreed 
with  the  chosen  response  for  7  of  the  last  10  responses.  At  this  point, 
the  operator  can  use  the  utility  values  generated  by  the  model  to  choose 
the  appropriate  response.  ADDAM  also  uses  the  concept  of  "environmental 
states"  to  correct  for  changes  in  the  decision  environment.  Thus,  as 
environmental  states  change,  the  parameters  of  the  model  change  and 
different  decisions  are  predicted.  This  requires  a  separate  "training 
program"  for  each  environmental  state.  The  next  section  will  demon¬ 
strate  how  this  model  works  for  the  ASW  environment. 


A SW  Jui v ironment 

The  ASW  problem  involves  a  TACCO  who  must  choose  the  number  and 
configuration  of  sonobouy  patterns  to  drop  at  specific  locations  in 
order  to  track  suspected  enemy  submarines.  Four  attributes  of  the 
possible  configuration  choices  were  considered  important:  detection 
index  [A^|  (i.e.,  probability  of  detecting  submarine),  area  covered  by 

the  configuration  [ A2 ] *  uncertainity  reduction  [ ] ,  and  resource  con¬ 
servation  [A^]  (i.e.,  number  of  sonobouys  that  are  available).  Figure  6 

shows  that  various  levels  of  these  attributes  are  associated  with  the 
possible  configurations  the  TACCO  can  choose.  Thus,  each  response 
(i.e.,  possible  configuration)  has  a  utility  vector  consisting  of  the 
levels  of  the  attributes  appropriate  for  that  response  [e.g.  ,  P^  = 

( fA 1 ,  ^2*  ^A^,  ^A^)  where  ^A^  is  the  appropriate  detection  index  level 

for  P j 1 .  The  relative  weighting  of  each  attribute  is  changed  after  each 

response  which  is  different  than  the  response  predicted  by  ADDAM.  A 
numerical  example  of  the  training  algorithm  and  the  utility  model  might 
help  In  understanding  the  procedures.  Before  any  responses  are  made  the 
utility  weights  for  each  attribute  are  1  (U^ ,  ll^,  U^,  U^) .  The  attri¬ 
bute  levels  for  each  response  pattern  are  set  and  normalized.  They  are 
normalized  so  the  range  of  values  Is  the  same  for  each  attribute.  For 
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FIGURE  6.  The  Mapping  of  Attribute  Levels  to 
Particular  Responses. 


27 


*  ■». a  . 


ADJUSTED  PREVIOUS 

WEIGHT  WEIGHT 


ADJUSTED 

FACTOR 


ATTRIBUTE 
VECTOR  OF 
CHOSEN 
PATTERN 

C*i 


ATTRIBUTE 
VECTOR  OF 
PREDICTED 
PATTERN 


X 


NWC  TP  6171 


instance,  In  our  example  it  will  be  assumed  that  the  values  tor  each 
attribute  are  transformed  so  that  the  levels  range  from  1  to  10  regard¬ 
less  of  their  underlying  scale.  To  make  our  example  simple,  we  will 
assume  there  are  only  two  possible  configurations,  and  P2  with  their 

levels  set  at  (4,  1,  9,  1)  for  and  P^  =  (6,  7,  1,  7).  Using  Equation 

1,  with  equal  unit  weights  for  each  attribute,  the  utility  for  Pj  =  19 

and  P2  *  21.  These  values  would  be  shown  the  operator  and  P2  would  bo 

the  predicted  response.  However,  if  the  operator  chose  Pj  instead,  then 

the  utility  weights  would  change,  reflecting  the  fact  that  the  operator 
apparently  weighed  attribute  1  more  heavily  than  the  other  attributes  in 
making  his  choice.  Since  the  changes  in  the  utility  weights  should  be  a 
gradual  process  reflecting  a  number  of  training  reiterations,  the  train¬ 
ing  adjustment  factor  in  Figure  7  (V)  is  set  at  0.04.  Using  this  train¬ 
ing  algorithm,  the  weights  for  the  four  attributes  would  change  to  0.92, 
0.84,  1.32,  and  0.84  which  would  cause  the  utility  values  lor  J'j  and  P0 


to  change  to  20.60  and  18.60,  respectively.  It  should  be  emphasized 

that  attribute  level  values  ( ,A, . . . .A. . . .  A  )  do  not  change;  rather,  the 

1  1  1  j  m  n  ” 

weighting  constants  for  the  attributes  in  Kquation  1  change  (U j . . . U . . . . U^) . 


Thus,  the  chosen  response  modified  the  utility  values  derived  from 
Equation  1  through  the  use  of  the  training  algorithm.  In  particular, 
the  modification  changed  the  predicted  response  pattern  from  to  P  . 


Figure  8  shows  a  computer  simulation  using  the  ADDAM  model  and  the 
number  of  decisions  made  before  the  values  stabilized  for  3  and  5  attri¬ 
butes.  These  figures  illustrate  that  stabilization  depends  on  the 
complexity  of  the  problem  (i.e.,  number  of  attributes  considered). 

However,  these  particular  values  stabilized  pretty  well  at  less  than  30 
decisions  for  complex  situations  and  less  than  20  for  simpler  ones. 

Thus,  at  least  for  these  simple  environments  simulated  in  the  labora¬ 
tory,  the  model  shows  promise  for  applications  where  a  great  number  of 
decisions  must  be  made  for  relatively  static  environments.  In  the  real 
world,  the  decision  environment  is  apt  to  change  rapidly  under  certain 
conditions.  The  ADDAM  adapts  to  rapid  changes  by  using  a  number  of 
separate  utility  weight  vectors  depending  on  the  conditioning  elements. 
Thus,  if  the  sea  state  changes  from  calm  to  rough,  the  ADDAM  automatically 
uses  a  different  training  algorithm,  resulting  in  different  utility 
weights.  In  this  way  various  environmental  states  are  matched  to 
various  ADDAM  models  and  adaption  can  be  made  rapidly  to  changes  in  the 
environment  that  are  predictable.  Also,  if  the  operator  realizes  that 
none  of  the  ADDAM  models  3re  appropriate  he  can  be  trained  to  rely  on 
hig  own  judgment  until  the  training  algorithm  stabilizes  to  reflect  the 
new  situation.  The  crucial  point  is  that  the  ADDAM  system  does  not 
automatically  make  decisions  but  rather  (after  a  sufficient  training 
period)  gives  the  operator  a  model  of  his  past  decisions.  Thus,  the 


29 


u*..  <■**. .  . 


NWC  TP  6171 


operator  Is  able  to  monitor  and  change  the  basic  strategy  as  the  situa¬ 
tion  dictates  and  to  use  the  ADDAM  aid  for  those  situations  when  decision¬ 
making  becomes  a  repetitive  task. 


Aid  Characteristics 


The  aid  described  above  is  only  one  version  of  the  ADDAM  aiding 
approach.  Other  aiding  approaches  involved  queries  for  an  information 
system  in  a  command  and  control  environment,  and  optimal  paths  for  an 
RPV.  Also,  some  of  the  other  versions  were  more  complex.  They  used 
both  Equations  1  and  2  for  action  selection  criteria.  The  TACCO  aid 
uses  only  Equation  1  and  does  not  consider  the  implications  of  more  than 
one  outcome  per  response.  However,  the  basic  aiding  approach  and  weak¬ 
nesses  were  very  similar  for  all  versions  of  the  aid.  The  main  weakness 
is  the  lack  of  research  done  in  an  operational  environment  to  test  the 
aid.  Laboratory  simulations  have  shown  the  aids  to  be  useful  in  a 
number  of  combat  environments.  However,  these  simulations,  by  their 
very  nature,  are  not  real  tests  of  the  system  in  their  actual  environ¬ 
ment.  The  system  must  be  tested  in  an  operational  environment  before 
any  meaningful  evaluations  can  be  made.  This  is  particularly  true 
because  the  system  is  unique  and  baseline  data  from  similar  operational 
systems  cannot  be  used  to  predict  performance. 

Also,  there  are  technical  problems  with  the  algorithm  itself.  The 
particular  MAUT  additive  model  discussed  previously  is  not  technically 
appropriate  for  the  ASW  environment.  This  is  because  the  attributes 
used  are  highly  correlated,  a  condition  which  violates  the  assumptions 
of  MAUT  models. 2,1 5  However,  other  research  has  found  that  the  predic¬ 
tive  power  of  such  models  is  high  even  when  some  of  their  basic  assump¬ 
tions  are  violated.21”25  The  model  used  is  simple  and  has  face  validity. 
The  next  step  is  to  examine  more  complex  models  or  even  simpler  models 
(perhaps  by  reducing  the  number  of  attributes) .  Then  the  competing 
models  can  be  compared.  It  is  even  possible,  that  a  preset  weighting  of 
various  attributes  based  on  an  average  of  expert  judgments  might  out¬ 
perform  any  model  generated  with  the  training  algorithm. 

In  summary,  the  ADDAM  approach  is  promising  for  military  environ¬ 
ments  in  which  a  large  number  of  repetitive  decisions  must  be  made. 
However,  the  actual  usefulness  of  such  an  approach  for  operational 
situations  has  not  been  thoroughly  evaluated;  more  tests,  both  in  the 
field  and  the  laboratory  must  be  conducted. 


24Edwards,  W.  Computer-Aided  Decision  Making:  A  Short  Course. 

Class  lecture,  UCLA,  Spring  1978. 

25Gardiner,  P.  C.  and  W.  Edwards.  "Public  Values:  Multi-Attribute 
Utility  Measurement  for  Social  Decisions,"  Proceedings  of  the  1975 
International  Conference  on  Cybernetics  and  Society ,  San  Francisco,  CA. 
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ADDAM  TAXONOMIC  BREAKDOWN 


Decision  Task9 

The  main  Casks  addressed  were  worth  assessment  and  action  selection. 
The  worth  assessment  model  resulted  in  a  multi-attribute  interval  scale. 
The  action  selection  suggestions  which  are  displayed  to  the  operator  are 
based  on  both  Equations  1  and  2  depending  on  the  particular  version  of 
ADDAM  used. 


Conditioning  Elements 

The  conditioning  elements  were  handled  in  two  ways.  A  "situation 
mask"  approach  consisted  of  introducing  dummy  variables  into  the  linear 
models  (Equations  1  and  2)  which  had  a  value  of  "one"  if  a  particular 
environmental  state  existed  and  "zero"  otherwise.  On  the  other  hand,  the 
"environmental  state"  approach  consisted  of  using  a  different  linear 
model  for  each  of  the  possible  16  "states-of-the-world".  For  this 
approach,  the  "conditioning  elements"  determined  the  linear  model  used 
for  a  particular  exercise. 


Mathematical  Models 

Two  mathematical  approaches  were  used:  MAUT  and  pattern-recognition 
models.  The  first  approach  consisted  of  linear  models  (Equations  1  and 
2)  to  rank  order  the  various  possible  actions  in  terms  of  relative 
value.  The  second  approach  was  used  to  update  the  utility  weights  in 
Equation  1  as  a  function  of  the  operator's  past  response  pattern. 
Attributes  favored  by  the  operator's  pattern  were  given  relatively  heavy 
weights  whereas  non-favored  attributes  were  given  low  weights.  The 
actual  weights  were  determined  by  a  "training  algorithm". 
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ASTDA :  ANALYTICS  INC. 

ASTDA  uses  a  number  of  displays  intended  to  improve  shipboard 
planning  of  air  strike  timing  for  air-to-ground  missions.  The  aid  is 
part  of  a  general  framework  of  aiding  procedures  which  gives  the  operator 
complex  information  in  an  understandable  format  using  nomograpic  display 
techniques.  Since  the  mathematical  model  underlying  ASTDA  is  particular 
to  the  air  strike  timing  problem,  this  report  will  concentrate  on  the 
overall  nomographic  technology.  One  of  the  advantages  to  the  ASTDA 
approach  is  that  both  "outcome  calculator"  models  and  "state"  models  can 
be  addressed.  The  distinction  between  these  two  models  will  be  dis¬ 
cussed  in  a  later  section. 


ASTDA  Scenario 


Figure  9  outlines  the  scenario  ASTDA  was  designed  to  aid.  The 
commander  of  an  aircraft  carrier  must  decide  when  to  launch  an  attack 
against  Onroda  Island.  Onroda  Island  has  about  120  planes  and  37  SAM 
and  AA  batteries  for  defense,  whereas  the  attack  force  has  12  A-6,  24 
A-7,  and  18  F-14  available  for  the  air  strike.  The  following  systems 
variables  are  considered  important: 
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FIGURE  9.  Air  Strike  Timing  Scenario. 


1.  Mission  segment 

2.  Fighters  and  attack  force  readiness  (own) 

3.  Enemy  force  readiness  at  target 

4.  Enemy  ground  force 

5.  Weather  at  target 

6.  Weather  at  carrier 

The  purpose  of  the  nomographs  is  to  display  as  much  pertinent 
information  as  possible  to  the  decision-maker  without  overloading  him. 


Nomographic  Displays 

A  nomographic  display  depicts  the  effects  of  two  or  more  indepen¬ 
dent  variables  on  the  performance  of  some  criterion  variable.  The 
display  shows  the  decision-maker  the  relationships  among  a  number  of 
different  variables  which  influence  systems  performance  allowing  him  to 
both  structure  the  decision  situation  and  to  choose  among  possible 
alternatives.  Table  4  lists  10  displays  for  the  air  strike  environment 
showing  both  independent  systems  variables  and  the  criterion  variable. 
The  criterion  variable  is  some  measure  of  systems  performance,  whereas 
the  independent  variables  are  those  likely  to  influence  the  air  strike 
mission's  success.  Three  points  should  be  made  concerning  these  dis¬ 
plays:  (1)  utility  is  the  most  common  criterion  variable;  (2)  most  of 
the  criterion  variables  are  a  function  of  air  strike  launch  time  and 
some  other  variable;  (3)  some  of  the  displays  give  range  and  variance 
estimates  as  well  as  showing  the  mean  effect  of  the  independent  variable! 
on  the  criterion. 

Utility,  although  defined  precisely  in  a  mathematical  sense,  is 
often  difficult  to  assess  except  in  relative  terms.  This  is  because  the 
scale  does  not  represent  a  physical  measurement  but  rather  attempts  to 
elicit  worth  values  from  the  operator  concerning  particular  outcomes.27 
The  elicitation  of  these  values  is  difficult18  and  often  does  not  repre¬ 
sent  the  true  preference  structure  of  the  operator.  The  reports  dis¬ 
cussing  ASTDA  did  not  go  into  much  detail  on  eliciting  utility  values 
and  it  must  be  assumed  that  such  questions  will  be  addressed  in  future 
developmental  work. 

As  Table  4  indicates,  the  main  purpose  of  the  ASTDA  displays  is  to 
depict  the  effects  of  various  systems  variables  (i.e. ,  weather,  number 
of  aircraft,  etc.)  as  a  parametric  function  of  strike  time.  Figure  10 
illustrates  the  type  of  formats  used  to  display  the  nomographs.  Figure 
10A  graphs  the  effects  of  number  of  A- 7  aircraft  on  utility  with  strike 
time  as  a  parameter  (the  ASTDA  displays  also  uses  color  to  code  strike 
time  differences).  Figure  10B  shows  the  average  effect  of  strike  time 
on  utility,  but  also  shows  confidence  limits  bounding  the  mean  effects. 


TABLE  4.  Nomographic  Displays  for  Strike  Time  Problem 
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FIGURE  10.  Examples  of  Possible  ASTPA  Display  Configurations 
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Although  this  information  could  be  used  to  aid  the  operator  in 
alternative  selection,  the  main  advantage  in  giving  the  decision-maker 
such  detailed  information  is  to  simplify  problem  structuring  (e.g., 
Table  4) .  These  displays  are  an  attempt  to  reduce  the  mass  of  informa¬ 
tion  inherent  in  a  combat  situation  to  manageable  levels  so  that  the 
decision-maker  can  structure  the  scope  of  the  decision  problem  in  his 
own  mind  before  choosing  among  alternative  strike  times. 

One  of  the  features  of  ASTDA  approach  is  the  "outcome  calculator". 
This  is  an  attempt  to  circumvent  the.  problems  associated  with  utility 
measures  by  modeling  the  combat  environment  so  that  the  criterion  value 
is  some  physical  measure  of  systems  performance.  The  relationship  of 
the  outcome  calculator  to  decision  theory  is  discussed  next. 


"State* '  versus  "Out come  Calculator"  Model _s 

Figure  1  shows  the  usual  "state"  model  approach  to  structuring  the 
decision  problem.  First,  the  decision-maker  lisLs  distinct  actions  and 
states-of-the-wor Id .  Next,  he  generates  a  utility  value  for  each  out¬ 
come  evaluating  directly  the  effects  of  the  occurrence  of  non-overlapping 
states-of-the-world  given  a  particular  action.  A  "state"  model  is 
inconvenient  for  two  reasons:  (1)  states-of-the-world  must  take  on 
discrete  values;  (2)  outcomes  may  be  uninterpretable  in  some  cases. 

For  example,  it  may  he  very  difficult  to  measure  the  utility  of  attack¬ 
ing  at  9:00  am  when  the  weather  is  poor  and  the  enemy  is  prepared, 
because  "poor  weather"  and  "enemy  preparation"  are  not  quantitatively 
def ined . 

An  "outcome  calculator”  is  a  mathematical  model  which  uses  opera¬ 
tional  research  techniques  to  compute  the  consequences  of  an  outcome  in 
terms  of  physical  criteria.  Tn  the  above  example,  an  outcome  calculator 
would  use  values  for  visibility,  probability  of  precipitation,  number  of 
enemv  planes  and  ground  defense  installations  as  input  variables.  The 
model's  equations  would  then  predict  the  extent  of  damage  to  both  the 
attack  and  enemy  forces.  Using  an  outcome  calculator,  the  commander 
could  make  decisions  in  terms  of  physically  meaningful  criteria  and  not 
have  to  generate  utility  values.  Also,  an  outcome  calculator  could  be 
used  to  generate  utilities.  In  this  case,  the  utility  values  would  be 
generated  from  the  "outcome  calculation"  values  using  MAUT  models  dis¬ 
cussed  previously  (e.g.,  12  friendly  aircraft  destroyed,  6  enemv  targets 
destroyed  =  II  utility  units).  There  are  a  number  of  disadvantages  to 
an  "outcome  calculator":  (!)  the  model  is  usually  mathematical Iv 
sophisticated,  incorporating  assumptions  which  are  valid  onlv  if  the 
relationship  among  the  model's  parameters  are  well  understood;  (2)  the 
model  is  situation  specific  —  if  parameters  not  included  in  the  model 
are  important  (i.e.,  enemy  morale,  new  weapons...),  then  the  model  is 
likely  to  make  inaccurate  predictions.  Thus,  an  outcome  calculator  is 
only  practical  for  situations  that  are  readily  modeled  and  are  important 
enough  to  warrant  the  effort. 
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ASTDA  TAXONOMIC  BREAKDOWN 

Decision  Tasks.  The  main  decision  tasks  addressed  are  problem 
structuring  and  information  selection.  In  a  combat  environment,  the 
number  of  variables  and  inherent  uncertainty  makes  the  structure  of  the 
decision  situation  extremely  complex.  The  nomographs  reduce  the  com¬ 
plexity  and  allow  the  decision-maker  the  opportunity  to  limit  the  scope 
of  the  decision  problem  to  manageable  levels.  In  particular,  the 
"outcome  calculator"  allows  the  decision-maker  to  quantify  outcomes  in 
order  to  evaluate  them.  This  reduces  the  number  of  possible  actions, 
since  some  possibilities  can  be  screened  out  without  resorting  to  more 
formal  decision  analysis.  Also,  the  displays  are  designed  to  depict 
concisely  the  most  valuable  information  in  the  decision  environment. 

Thus  the  information  selection  task  is  simplified  by  using  optimal 
information  displays. 

However,  the  "outcome  calculator"  and  displays  developed  by  Analytics, 
and  the  models  on  which  they  are  based  are  still  evolving.  They  have 
not  yet  been  evaluated  thoroughly  in  a  laboratory  situation  or,  even 
more  importantly,  used  in  an  actual  operational  environment.  Thus  all 
that  can  be  said  is  that  they  are  interesting  concepts,  which,  when 
developed,  may  prove  to  be  useful  tools. 

Conditioning  Elements.  Conditioning  elements  are  handled  two  ways 
by  the  ASTDA  aid.  For  state  models,  values  of  conditioning  elements 
define  particular  states-of-the-world.  For  example,  temperature  91°, 
visibility  13  miles,  enemy  forces  50%  prepared  would  define  one  state; 
qualitatively  it  would  be  S^  =  hot,  fair  visibility  and  enemy  prepara¬ 
tion  moderate.  For  the  "outcome  calculator"  these  variables  would  be 
used  as  inputs  to  the  "outcome  calculator"  model.  These  inputs  would  be 
used  to  solve  equations  to  predict  the  outcome  of  an  air  strike  in  terms 
of  own  and  enemy  forces  destroyed. 

Mathematical  Models.  The  ASTDA  approach  uses  decision  analytic  and 
traditional  operational  research  modeling  techniques.  The  general 
evolution  of  the  ASTDA  approach  is  away  from  "state"  models  and  towards 
"outcome  calculators".  The  mathematical  model  for  the  calculator  is 
complex,  using  a  number  of  parameters  built  into  the  algorithm  as  well 
as  a  number  of  input  variables.  The  distributional  aspects  of  the 
model's  parameters  are  considered  (mean,  standard  deviation,  skewness, 
and  type  of  distribution).  Monte  Carlo  methods  can  be  used  to  predict 
output  distributions  in  terms  of  own  and  enemy  losses. 

Decision  theoretic  modeling  is  used  in  the  ASTDA  approach,  especially 
in  Bayesian  updating  models  and  expected  utility  calculations.  However, 
the  trend  seems  to  be  moving  toward  giving  the  decision-maker  outcome 
data  in  terms  of  physically  meaningful  units  (weather,  number  of  air¬ 
craft  loss  etc.)  rather  than  in  utility  units. 
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SKETCH  MODEL:  INTEGRATED  SCIENCES  CORPORATION 

The  SKETCH  model  is  an  attempt  to  use  aiding  systems  in  which  "the 
man  helps  the  machine  help  the  man"  (Figure  11).  The  operator  performs 
tasks  which  aid  the  machine  by  limiting  the  decision  problem  and  the 
machine,  in  turn,  computes  optimal  solutions  which  feed  back  to  the 
operator  and  cause  him  to  further  structure  the  task.  The  most  important 
feature  of  the  aid  is  the  interactive  nature  of  the  tasks  that  are 
assigned  to  man  and  machine.  The  purpose  of  the  aid  is  to  help  an 
aircraft  commander  plan  his  flight  path  for  an  air  strike  mission.  Two 
mathematical  approaches  for  implementing  the  aid  have  been  investigated: 
non-linear  programming  and  dynamic  programming. 


Scenar io 


The  aid  helps  the  commander  find  optimal  paths  for  air  strikes 
against  the  airport  on  Onroda  Island.  Figure  12  shows  an  example  of  a 
SKETCH  display.  The  crosses  are  radar  installations,  whereas  the  con¬ 
centric  circles  around  the  island  represent  isocontours  indicating 
locations  with  equal  probability  of  detection  by  the  rad.: .  The  com¬ 
mander's  task  is  to  choose  a  flight  path  which  minimizes  both  probability 
of  detection  and  fuel  consumption. 


Interactive  Aiding 

The  operator  visual Iv  examines  the  display  to  choose  likely  avenues 
of  approach.  Hi*  then  marks  the  candidate  flight  paths  on  the  SKETCH 
display  in  the  following  manner  (Figure  12).  He  generates  six  points 
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I.  UNAIDED  DECISION-MAKING 
(Man  Only) 

human] _ °EC  1510,1 - ►  ACTION 


II.  hachine-aioeo  decision-making 
(Machine  Helping  Man) 


III.  HUMAN-AIDED  MACHINE  DECISION-RECOMMENDING 
(Man  Helping  Machine  to  Help  Man) 


HUMAN 


HUMAN  DEC  I S I  ON 


ACTION 


HUMAN-AIDING 


MACHINE  RECOMMENDATION  |MArii’,u7 


FIGURE  11.  Three  Strategies  for  Decision-Aiding. 
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spaced  at  equal  distances  to  determine  the  flight  path.  Actually,  two 
of  these  points  are  predetermined,  since  the  first  and  last  points  are 
the  starting  point  and  Onroda  Island,  respectively.  The  other  four  are 
referred  to  as  part  "way"  points  for  the  proposed  flight  path  (i.e., 
connecting  all  six  points  defines  the  flight  path).  The  computer 
examines  these  points  using  non-linear  programming  optimization  techniques. 
The  program  attempts  to  minimize  a  trade-off  function  between  fuel 
consummed  and  probability  of  detection.  Thus  either  flying  directly 
toward  the  airport  (marked  with  an  X)  or  flying  around  all  high  proba¬ 
bility  contours  does  not  necessarily  represent  an  optimal  flight  path  in 
terms  of  the  trade-off.  The  non-linear  programming  algorithm  (cf., 

Walsh  and  Schecterman,  1979,  Appendix  B)  examines  points  near  the  chosen 
"way"  points  in  order  to  find  a  more  optimal  path.  The  process  is 
reiterative  in  nature  with  different  points  being  chosen  until  either 
the  improvement  fails  to  surpass  some  minimal  criterion  or  a  pre-set 
time  has  elapsed.  The  result  is  that  an  optimal  path,  in  the  general 
vicinity  of  the  path  chosen  by  the  operator,  is  displayed  on  the  SKETCH 
model  display.  The  trade-off  function  also  is  used  to  display  a  utility 
value  for  that  particular  path. 

The  operator  then  chooses  a  path  in  a  different  region  and  the 
computer  finds  the  optimal  path  in  that  general  neighborhood  and  displays 
the  results  to  the  operator.  The  final  result  is  that  the  operator  can 
view  optimal  paths  for  a  number  of  different  regions  of  the  SKETCH  dis¬ 
play  with  a  measure  of  merit  given  for  each  path.  The  aiding  system 
attempts  to  allocate  to  both  man  and  machine  tasks  they  are  best  suited 
for.  The  operator  is  responsible  for  global  strategy,  choosing  a  number 
of  likely  flight  paths  based  on  his  own  experience  and  the  displayed 
contours.  The  computer,  in  effect,  does  the  fine  tuning  by  choosing  the 
best  paths  in  the  general  area  picked  by  the  operator.  The  aid  also 
computes  a  measure  of  merit  based  on  the  trade-off  function.  This  is 
what  is  meant  by  the  "man  helping  the  machine  help  the  man."  The  operator's 
ability  to  see  a  number  of  plausible  strategies  aids  the  machine  by 
restricting  the  amount  of  computation  necessary.  The  computer,  in  turn, 
does  fine  tuning  and  computations  which  are  beyond  the  scope  of  the 
operator's  capabilities.  Besides  the  non-linear  algorithm  discussed, 
the  aid  can  also  use  dynamic  programming  algorithms.  A  brief  descrip¬ 
tion  and  comparison  of  the  two  methods  follow. 


Non-Linear  and  Dynamic  Programming  Techniques 
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The  form  of  the  equation  driving  the  non-linear  programming  algorithm 
is  given  below. 


where 


U  =  f  (X....X....X  ) 

1  J  n 

a 

Xj  is  an  independent  variable. 

f (X, . . .X. . . .X  )  is  a  function  defined  on  these  variables. 

1  J  n 

U  is  a  utility  value  which  is  maximized  when  the  trade-off 
between  fuel  and  probability  of  detection  is  minimized. 


The  13  variables  of  interest  include  both  the  longitudinal  and 
latitudinal  locations  of  the  four  "way"  points  and  the  airspeeds  for 
each  of  five  legs  of  the  flight  path.  A  trial  is  the  changing  of  the 
value  of  one  of  the  variables.  The  effect  of  each  trial  on  U  determines 
the  next  step  in  the  algorithm.  For  the  first  trial,  the  value  of  the 
variable  changes  an  arbitrary  amount  E.  If  U  increases  or  stays  the 
same,  E  itself  increases  on  the  next  trial.  Thus,  every  time  a  value 
change  either  does  not  affect  or  increase  U,  its  value  in  the  next 
trial  increases  at  a  greater  rate.  However,  whenever  a  trial  results 
in  decreasing  U,  the  value  of  the  variable  is  set  and  the  next  variable 
of  interest  enters  into  the  algorithm.  After  all  13  variables, are 
examined,  the  algorithm  repeats  the  process  until  the  value  of  U  increases 
by  less  than  1%  or  a  time  limit  criterion  expires. 


The  dynamic  programming  algorithm  uses  similar  logic.  However,  the 
size  and  direction  of  value  changes  are  determined  by  predefined  rules. 

The  SKETCH  display  is  overlaid  by  a  grid  system  (for  example,  a  16  x  16 
grid  would  entail  256  points  on  the  display  and  every  possible  flight  path 
would  have  to  go  through  these  points).  Figure  13  shows  an  allowable 
point  change  in  the  flight  path  that  passes  through  point  A.  Thus,  the 
flight  path  can  change  point  by  point  in  order  to  maximize  U.  Also,  for 
each  change  in  flight  path,  the  maximization  function  considers  the 
effects  of  three  airspeeds  (high,  medium,  and  low).  The  process  of 
constructing  a  flight  path  entails  the  algorithm  considering  every  legal 
transition  from  point  to  point,  combined  with  each  of  the  three  air¬ 
speeds.  Thus,  if  the  grid  system  is  large,  the  number  of  reiterations 
necessarv  to  find  an  optimal  path  would  be  large  and  time-consuming, 
especially  for  the  computers  available  for  the  SKETCH  aid.  The  operator 
can  aid  the  computer  by  limiting  the  dynamic  programming  to  regions  in 
the  display  that  would  contain  likely  flight  paths  and  by  using  a  coarse 
(i.e.,  one  having  few  points)  grid  system  for  initial  determination  of 
optimal  flight  paths.  For  subsequent  reiterations,  the  algorithm  could 
be  refined  by  limiting  the  regions  further  and  increasing  the  density  of 
the  grid  system  (it  should  be  obvious  that  the  more  points  in  the  grid 
system,  the  more  accurate  the  algorithm).  Thus,  again,  the  operator  is 
helping  the  machine  to  help  the  man.  In  this  case,  he  would  aid  the 
machine  by  limiting  the  scope  of  the  dynamic  programming  to  areas  that 
are  likely  to  contain  the  optimal  paths.  The  next  section  is  a  review 
of  experimental  work  done  using  variations  of  the  SKETCH  model. 
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FIGURE  13.  Allowable  Transitions  From  Point  A  on  a 
5  by  5  Grid  Using  the  Dynamic  Programming  Algorithm. 
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Experimental  Comparisons 

The  SKETCH  model  was  evaluated  in  two  experiments  using  three  main 
experimental  conditions:  man  alone,  man-machine  combinations,  and 
machine  alone.  Also,  the  two  types  of  programming  were  considered  in 
separate  experiments.  For  both  experiments,  the  initial  path  chosen  by 
the  operator  was  the  estimate  of  man  alone.  The  man-machine  combination 
consisted  of  the  type  of  interactive  aiding  described  in  the  previous 
two  sections.  A  number  of  different  runs  using  the  computer  to  solve 
the  problem  automatically  were  done  to  measure  the  performance  of 
machine  alone.  The  results  for  the  non-linear  programming  are  shown  in 
Figure  14.  The  figure  suggests  that  the  man-machine  system  out-performs 
either  the  man-alone  or  the  machine-alone  system.  The  man-alone  results 
are  somewhat  confounded,  because  they  are  estimates  of  performance  at  the 
beginning  of  the  session,  and  it  might  be  argued  that,  if  the  operator 
took  longer,  his  results  would  be  the  same  as  the  man-machine  system. 
However,  Figure  14  shows  that  the  man-machine  system  found  a  nearly 
optimal  path  at  approximately  3  minutes  into  the  problem,  strongly 
suggesting  the  advantages  of  using  the  man-machine  system.  By  compari¬ 
son,  both  machine-only  systems  were  inferior  to  the  man-machine  system. 

The  dynamic  programming  data  came  from  a  different  experiment  and 
could  not  be  directly  compared  to  the  non-linear  results.  However,  the 
experimenters  concluded  for  a  number  of  reasons  that  the  non-linear 
programming  algorithm  was  superior.  Thus,  it  appears  that  future  work 
using  the  SKETCH  model  will  concentrate  on  the  non-linear  programming 
algorithm  using  the  man-machine  mode. 


SKETCH  TAXONOMIC  BREAKDOWN 

Decision  Tasks.  The  non-linear  programming  algorithm  computes  the 
utility  weights  for  various  flight  paths.  However,  the  uncertainties 
(isocontours  in  Figure  12)  involved  in  choosing  the  paths  are  considered 
in  the  utility  function.  Thus,  the  utility  values  are  both  worth  assess¬ 
ments  and  action  selection  criteria.  Furthermore,  the  algorithm  actually 
generates  possible  actions  (i.e.,  possible  paths).  Thus,  the  aid  does 
some  of  the  listing  of  actions  subsumed  in  the  problem  structuring  task. 
However,  most  of  the  problem  structuring  is  done  by  the  operator,  since 
the  aid  only  looks  for  better  paths  in  the  general  vicinity  of  those 
chosen  by  the  operator.  The  display  depicts  probability  estimations 
using  isocontours.  However,  the  actual  values  are  supplied  as  a  result 
of  operational  and  intelligence  inputs. 


Cumulative  Normalized  Utility 
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Time  Into  Problem  (Minutes) 

FIGURE  14.  Results  From  the  Man-Alone,  Man-Machine  and  the 
Two  Automatic  (Machine-Alone)  Conditions  in  the  Non-Linear 
Programming  Experiment. 
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Conditioning  Elements 

One  of  the  weaknesses  of  the  aid  is  the  use  of  fuel  consumption  and 
probability  of  detection  as  the  only  conditioning  elements.  The  util  ity 
function  incorporating  these  two  elements  is  explicitly  outlined  in 
Walsh  and  Schecterman  (1978).  Basically,  it  is  a  non-linear,  fairly 
complex  trade-off  function  which  is  not  obvious  to  the  operator  and  thus 
the  computation  done  by  the  aid  would  overload  the  operator  in  a  normal 
planning  environment.  However,  conditioning  elements  such  as  weather, 
number  of  aircraft,  force  at  Onroda  Island,  etc.,  would  have  to  be  con¬ 
sidered  before  any  realistic  assessment  of  flight  paths  could  be  made. 
This  only  points  out  that  the  SKETCH  model  is  still  basically  an  experi¬ 
mental  concept.  Its  use  in  an  operational  environment  would  entail 
major  revisions  to  insure  that  the  aid  is  useful  in  a  realistic  environ¬ 
ment  . 


M a  them atical  Model s 

Non-linear  programming  and  dynamic  programming  algorithms,  popular 
as  economic  and  operational  research  tools,  are  the  basis  of  the  models 
used  by  the  SKETCH  aid.  The  non-linear  techniques  seem  to  be  emerging 
as  the  approach  the  developers  of  the  aid  will  use  in  future  research 
efforts. 
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EWAR:  DECISION-SCIENCE  APPLICATIONS,  INC. 

The  EWAR  devices  are  a  collection  of  displays  intended  to  aid  the 
operator  in  choosing  optimal  radar  configurations  which  maximizes  surveil¬ 
lance  capabilities  while  minimizing  identification  from  enemy  aircraft 
because  of  radar  emissions.  The  displays  range  from  simple  displays 
giving  information  concerning  a  single  system  parameter  to  complex 
displays  depicting  trade-off  functions  between  surveillance  and  security. 
Most  of  the  displays  are  not  designed  to  show  the  operator  one  optimal 
course  of  action.  Instead,  the  aids  allow  the  operator  to  explore  the 
advantages  and  disadvantages  of  using  various  radar  configurations.  The 
individual  aids  are  sequenced  in  a  manner  designed  to  increase  the 
operator's  insight  into  various  aspects  of  the  combat  situation.  They 
suggest  alternatives  he  might  not  have  considered  without  the  aids. 


Scenario 


The  general  combat  scenario  involves  a  task  force  consisting  of  an 
aircraft  carrier  (CV),  a  cruiser  (CA),  and  two  smaller  ships.  The 
electronic-warfare  (EW)  officer  must  develop  a  plan  for  radar  surveil¬ 
lance.  The  EW  officer  knows  that  there  is  a  high  probability  of  an 
enemy  missile  attack.  His  problem  is  that  he  must  be  able  to  track  as 
many  of  the  missiles  as  possible  on  the  radar.  However,  the  emissions 
from  the  radar  give  distinct  signatures  that  inform  the  enemy  of  the 
type  of  radar  coming  from  each  ship.  It  is  assumed  that  the  enemy  knows 
the  location  of  the  task  force,  but  cannot  identify  each  ship.  Thus,  if 
eight  missiles  were  available,  the  enemy  commander  would  be  forced  to 
target  two  missiles  at  each  ship  in  the  task  force.  This  would  mean 
that  the  carrier  would  have  a  good  probability  of  surviving  an  attack 
especially  if  the  carrier  had  adequate  radar  surveillance.  Unfortunately, 
the  more  radar  coverage  the  CV  elects  to  use,  the  more  precisely  the 
enemy  can  determine  its  locations.  If  the  enemy  knew  exactly  where  the 
CV  is  located,  then  he  would  target  all  eight  missiles  at  the  carrier, 
making  it  very  vulnerable.  Figure  15  illustrates  two  possible  radar 
configurations  the  EW  officer  might  consider.  The  first  configuration 
uses  all  radar  equipment  on  all  ships.  The  radars  are  listed  under  the 
letter  designations  (e.g.,  under  A  are  SPS-10  ...  SPS-52)  and  each  ship 
has  a  unique  pattern  of  signatures.  Thus,  the  enemy  commander,  based  on 
intelligence  information,  could  pinpoint  the  type  of  ship  at  each 
location  and  make  targeting  decisions  accordingly.  On  the  other  hand, 
the  enemy  commander  would  he  able  to  determine  the  exact  location  of 
only  the  least  valuable  ship  for  the  second  proposed  radar  configuration. 
The  EW  officer  must  determine  the  cost  of  reducing  radar  surveillance 
for  the  second  plan  and  weigh  this  against  the  benefit  of  denying  the 
enemy  commander  information  about  particular  ship  locations. 
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RADARS 
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CONFIGURATION  1:  ALL  RADARS  ON 


RAOARS  SPS-10 
SPS— 43 


RADARS  SPS-10 
SPS— 43 


RAOARS  SPS-10 


RADARS  SPS-10 
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CONFIGURATION  2:  TWO  RADARS  ON 


SHIPS  KEY 

A  -  AIRCRAFT 
CARRIER 

B  -  CRUISER 


C  •  DESTROYER 
D  -  MISC  SHIP 


FIGURE  15.  Display  Showing  Two  Possible  Radar  Configurations 
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EWAR  Displays 

There  are  three  objectives  for  EWAR  aids: 

1.  Compute  quantitative  information 

2.  Display  the  effects  of  EW  plans  on  both  surveillance  and 
security 

3.  Show  trade-off  functions 

The  first  group  of  displays  are  the  "order  of  battle”  displays.  These 
displays  give  the  EW  officer  information  concerning: 

1.  The  type  of  radars  possible  for  each  ship  in  the  task  force 

2.  The  relative  value  of  each  ship 

3.  Characteristics  of  the  threatening  missiles  (velocity,  alti¬ 
tude,  etc.) 

The  next  group  of  displays  give  information  concerning  surveillance. 
Isocontours,  depicting  probability  of  detecting  enemy  threats,  are  shown 
encircling  the  task  force  (Figure  16)  with  both  maximum  range  and 
cumulative  probability  information.  An  "outcome  calculator"  also 
figures  probable  damage  to  the  task  force  from  an  attack.  This  aid 
needs  inputs  regarding  the  type  of  surveillance  configuration  and  size 
and  type  of  the  enemy  threat.  The  calculation  assumes  the  enemy  has 
perfect  information  concerning  the  locations  of  the  ships  in  the  task 
force. 

The  next  series  of  displays  is  intended  to  show  the  effects  of 
turning  off  radars  in  order  to  deny  the  enemy  information.  For  example, 
the  "all  on"  configuration  in  Figure  15  gives  the  enemy  perfect  informa¬ 
tion  whereas  an  "all  off"  configuration  would  entail  maximum  uncertainty. 
The  information  denial  displays  show  confusion  matrixes  based  on  possible 
changes  in  radar  configurations.  For  example,  the  CV  Kitty  Haiok  would 
have  a  67%  probability  of  being  confused  with  a  cruiser  or  destroyer  for 
the  second  configuration  in  Figure  15.  Also,  the  probabilities  associated 
with  identifying  the  ships  for  a  particular  radar  configuration  plan  are 
used  to  compute  tire  perceived  value  of  the  ship. 


■  ,E,  P<W  h 

i=l 


where 

=  perceived  value  of  ship  k  under  the  proposed  plan 

P(Si/Sk)  =  the  probability  of  enemy  identifying  ship  as  being  i, 

. given  that  it  is  k 

A^  =  actual  numerical  value  assigned  to  ship  i  by  the  task 

force  commander 

n  =  number  of  ships  in  task  force 
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EMCON  PLAN 


FIGURE  16.  Example  of  a  Possible  Trade-Off  Map  for  the  EWAR  Displays 
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The  purpose  of  informational  denial  is  to  lessen  the  perceived 
value  of  high  value  ships,  while  increasing  the  perceived  value  of  low 
value  ships.  Thus,  the  operator  receives  independent  displayed  informa¬ 
tion  concerning  both  the  surveillance  and  information  denial  aspects  of 
the  radar  configuration  plans  he  is  considering.  The  last  set  of  dis¬ 
plays  is  an  attempt  to  combine  this  information  and  give  the  operator 
enough  information  to  make  a  trade-off  between  surveillance  and  informa¬ 
tion  denial  in  order  to  evaluate  possible  configurations. 

Figure  16  shows  a  trade-off  map.  The  surveillance  capabilities  are 
shown  as  isocontours.  The  information  denial  component  is  displayed  by 
the  size  of  the  bars  associated  with  the  number  representing  each  ship. 

The  top  bar  under  each  number  is  the  actual  value  of  the  ship  and  the 
bottom  bar  is  the  perceived  value.  The  ratio  between  the  two  gives  the 
operator  a  visual  cue  as  to  the  impact  of  a  particular  configuration  on 
information  denial.  Figure  17  shows  the  possible  trade-off  functions 
between  surveillance  and  information  denial.  Both  components  are  scaled 
by  a  "measure  of  merit"  between  0  and  1.0.  Zero  represents  both  no 
surveillance  and  complete  information  to  the  enemy  and  1.0  was  the 
opposite  for  both  cases.  The  graphs  show  how  the  combined  (i.e., 
additive)  value  of  these  two  components  changes  as  the  operator's 
weightings  for  information  denial  varies  (the  weights  for  surveillance 
are  1  -  Information  denial  weights).  The  cross-over  points  are  the 
place  where  one  configuration  plan  becomes  preferred  over  another.  Thus, 
th<  all-radar-on  is  preferred  to  the  all-radar-off  plan  until  information 
denial  has  a  0.6  weighting,  after  which  the  all-radar-off  plan  is  pre¬ 
ferred.  However,  for  the  middle  range  of  weights  (~0. 20-0. 70) ,  the 
plan  designated  current  is  preferred  to  either  of  the  other  two  plans. 

The  graph  does  not  show  the  optimal  solution  to  the  operator.  Rather, 
it  allows  the  operator  to  choose  his  own  weightings  for  the  two  com¬ 
ponents  (surveillance  and  security)  and  see  how  it  affects  the  proposed 
configuration  alternatives.  The  final  aid  in  the  KWAR  inventory  is 
another  "outcome  calculator".  The  previous  "outcome  calculator"  mentioned 
as  a  surveilianco.  aid  computed  the  amount  of  damage  done  to  the  Fleet, 
if  the  enemy  had  perfect  knowledge.  This  final  "outcome  calculator" 
recomputes  this  estimate  given  the  actual  information  value  and  surveil¬ 
lance  values  for  each  of  the  candidate  configuration  plans.  It  should 
be  noted  that  both  "outcome  calculators"  are  based  on  extremely  simple 
assumptions,  especially  compared  to  complex  models  such  as  the  ASTDA 
calculator  (e.g.,  given  a  .80  probability  of  detection,  5  of  7  dragon 
missiles  directed  towards  the  Kitty  ILiwk  would  impact,  causing  60% 
damage).  Simplicity  seems  to  be  the  hallmark  of  the  FWAR  aids.  This  has 
a  great  advantage  because  the  models  underlying  the  display  can  be 
explained  to  the  KW  officer  without  too  much  difficulty.  This  allows 
the  operator  to  build  up  an  accurate  picture  of  what  is  taking  place  and 
why.  Contrasted  with  some  of  the  earlier  aids  such  as  the  "triangle" 
aid,  this  group  of  aids  is  designed  more  to  develop  insight  than  to 
suggest  optimal  courses  of  action.  The  sequence  of  looking  at  the  aids 
structures  the  operator's  ability  to  understand  the  KW  problem.  First, 
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he  is  given  general  knowledge  about  the  combat  environment.  Next,  he 
looks  at  both  the  surveillance  and  information  denial  problem  in  isola¬ 
tion.  Only  when  he  understands  the  important  parameters  of  each  of 
these  problems  considered  separately  does  he  combine  this  information  in 
order  to  figure  a  trade-off  that  will  lead  to  an  optimal  configuration. 
However,  the  simplicity  of  the  aid  may  be  a  drawback  as  well.  It 
remains  to  be  seen  if  the  simple  algorithms  the  aids  are  based  on 
accurately  reflect  an  operational  environment.  Only  testing  in  this 
environment  could  determine  the  ultimate  usefulness  of  the  EWAR  system. 


EWAR  TAXONOMIC  BREAKDOWN 

Decision  Tasks.  The  aids  help  the  operator  mostly  in  worth  evalua¬ 
tion  and  information  selection.  The  tasks  are  well  structured  because 
the  operator  follows  steps  outlined  by  military  doctrine.  The  aids  help 
the  operator  understand  the  consequences  of  different  configurations  for 
both  surveillance  and  security.  This  information  helps  the  operator 
evaluate  the  worth  of  the  various  possible  plans.  In  a  manner  similar 
to  the  ASTDA  displays,  the  EWAR  displays  condense  a  great  deal  of  infor¬ 
mation  into  a  few  fairly  easily  understood  information  displays.  Further¬ 
more,  the  "outcome  calculators"  lead  to  action  selection  criteria. 

However,  the  operator  should  be  so  familar  with  the  military  situation 
because  of  the  other  displays  that  this  advisory  would  be  only  part  of 
the  data  upon  which  he  bases  his  decision. 

Conditioning  Elements.  A  few  conditioning  elements,  such  as  a  type 
of  threat,  are  used  as  input  values  when  computing  display  values. 

However,  a  number  of  important  conditioning  elements  for  the  EW  situa¬ 
tion  are  not  considered  by  the  aids  at  this  time  (e.g.,  weather,  enemy 
intelligence  resources,  enemy  tactics).  It  is  assumed  that,  when  the 
aids  are  introduced  into  an  operational  environment,  the  number  of 
conditioning  elements  considered  will  increase  considerably. 


Mathematical  Models 

Rather  than  following  any  particular  type  of  mathematical  pro¬ 
cedures  (e.g.,  decision  theoretic  techniques,  pattern  recognition 
modeling),  EWAR  aids  seem  to  be  developed  using  pragmatic  approaches  to 
modeling. 
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COMPARISONS 


Tables  5,  6,  and  7  serve  to  compare  the  aids  on  the  three  components 
of  the  taxonomic  breakdown.  Only  the  most  important  decisions  addressed 
are  shown  in  the  fifth  table.  The  modules  from  Decision  and  Designs  are 
not  shown  since  they  are  a  collection  of  aids,  some  of  which  were  still 
in  the  conceptual  stage  when  this  review  report  was  written.  Instead, 
the  best  developed  of  these  aids  (the  triangle  displays)  is  shown.  The 
number  of  checkmarks  for  each  aid  should  not  be  confused  with  a  measure 
of  merit,  since  the  aids  were  developed  for  quite  different  reasons. 


SUMMARY  AND  CONCLUSIONS 


Five  aiding  systems  developed  for  naval  combat  environments  were 
reviewed.  Particular  emphasis  was  placed  on  decision  tasks  addressed, 
how  conditioning  elements  were  accounted  for,  and  the  basic  modeling 
approach  on  which  the  aids  were  based.  A  number  of  general  trends  seem 
to  emerge  from  the  review. 

Determination  of  a  single  optimal  action  was  not  considered  the 
primary  purpose  of  most  aids.  Rather,  the  aids  tended  to  generate 
multiple  criteria  which  the  operator  could  use  to  structure  the  problem 
in  his  own  mind  before  making  a  decision. 

There  is  n£  one  general  methodological  approach  to  decision  aiding. 
Models  based  on  a  wide  variety  of  procedures  were  used  to  help  evaluate 
decisions.  Besides  the  more  traditional  decision  analytic  techniques 
such  as  Bayes  Thereom  and  utility  theory,  procedures  based  on  dynamic 
and  non-linear  programming,  pattern  recognition  models  and  "outcome 
calculations"  were  employed.  if  anything,  decision-aiding  is  best 
characterized  in  terms  of  the  diversity  of  approaches  used. 
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Most  of  the  aids  helped  the  operator  in  such  tasks  as  probability 
updating  and  worth  evaluation,  particularly  where  there  were  complex 
computations  or  memory  storage  problems.  For  the  most  part,  generation 
of  a  global  strategy  and  the  final  structuring  of  the  problem  were  left 
to  the  operator  or  commander.  However,  some  of  the  aids  enhanced 
information-gathering  by  using  optimal  formating  techniques.  These  aids 
displayed  the  most  important  systems  information  to  the  operator  in  a 
concise  form.  This  helped  the  operator  in  problem  structuring,  since 
the  displays  encourage  the  operator  to  eliminate  obviously  poor  choices. 
Also,  the  variety  of  information  sources  displayed  by  these  aids  could 
suggest  to  the  operator  actions  or  hypotheses  he  might  not  have  con¬ 
sidered  otherwise. 

On  a  more  negative  note,  there  is  little  evidence  that  these  aids 
would  be  useful  in  an  actual  operational  environment.  This  is  not  a 
criticism,  since  most  of  these  aids  were  originally  conceived  as  a 
research  tool.  However,  the  ultimate  test  of  any  decision  aid  is  whether 
it  can  help  the  operator  in  the  real-world.  Further  development,  labora¬ 
tory  experimentation,  and  field  tests  will  be  necessary  before  these 
aids  can  meet  this  requirement.  Preliminary  testing  at  the  University 
of  Pennsylvania  and  proposed  further  development  of  these  aids  suggest 
that  decision-aiding  may  prove  to  be  a  useful  tool  in  the  near  future. 
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